Ensemble-in-One: Ensemble Learning within Random Gated Networks for Enhanced Adversarial Robustness

Adversarial attacks have threatened modern deep learning systems by crafting adversarial examples with small perturbations to fool the convolutional neural networks (CNNs). To alleviate that, ensemble training methods are proposed facilitate better robustness diversifying vulnerabilities among sub-models, simultaneously maintaining comparable natural accuracy as standard training. Previous practices also demonstrate that enlarging can improve robustness. However, conventional poor scalability, owing rapidly increasing complexity when containing more sub-models in ensemble. Moreover, it is usually infeasible train or deploy an substantial tight hardware resource budget and latency requirement. In this work, we propose Ensemble-in-One (EIO), a simple but effective method efficiently enlarge random gated network (RGN). EIO augments candidate model replacing parametrized layers multi-path blocks (RGBs) construct RGN. The scalability significantly boosted because number of paths exponentially increases RGN depth. Then from numerous other within RGN, every path obtains Our experiments consistently outperforms previous smaller computational overheads, achieving accuracy-robustness trade-offs than under black-box transfer attacks. Code available at https://github.com/cai-y13/Ensemble-in-One.git